08. Calculate a Covariance Matrix

Calculate a Covariance Matrix

Remember how we defined the covariance matrix:

P=[Cov(rA,rA)Cov(rA,rB)Cov(rB,rA)Cov(rB,rB)]. \mathbf{P} = \begin{bmatrix} \mathrm{Cov}(r_A,r_A) & \mathrm{Cov}(r_A,r_B)\\ \mathrm{Cov}(r_B,r_A) & \mathrm{Cov}(r_B,r_B) \end{bmatrix}.

And covariance is

Cov(rA,rB)=E[(rA−rˉA)(rB−rˉB)]. \mathrm{Cov}(r_A,r_B) = \mathrm{E}[(r_A - \bar{r}_A)(r_B - \bar{r}_B)].

If rA r_A and rB r_B are discrete vectors of values, that is, they can take on the values (rAi,rBi) (r_{Ai}, r_{Bi}) for i=1,…,n i = 1,\ldots, n , with equal probabilities 1/n, then the covariance can be equivalently written,

=1n−1∑i=1n(rAi−rˉA)(rBi−rˉB). = \frac{1}{n-1}\sum_{i=1}^n(r_{Ai} - \bar{r}_A)(r_{Bi} - \bar{r}_B).

We use n−1 n-1 in the denominator of the constant for the same reason that we use n−1 n-1 in the denominator of the constant out front in the sample standard deviation—because we have a sample, and we want to calculate an unbiased estimate of the population covariance.

But if rˉA=rˉB=0 \bar{r}_A = \bar{r}_B = 0 , then the covariance equals

=1n−1∑i=1nrAirBi. = \frac{1}{n-1}\sum_{i=1}^nr_{Ai}r_{Bi}.

In matrix notation, this equals

1n−1rATrB. \frac{1}{n-1}\mathbf{r}_A^\mathrm{T}\mathbf{r}_B.

Therefore, if r \mathbf{r} is a matrix that contains the vectors rA \mathbf{r}_A and rB \mathbf{r}_B as its columns,

r=[⋮⋮rArB⋮⋮], \mathbf{r} = \begin{bmatrix} \vdots & \vdots\\ \mathbf{r}_A & \mathbf{r}_B\\ \vdots & \vdots \end{bmatrix},

then

rTr=[⋯rA⋯ ⋯rB⋯][⋮⋮ rArB ⋮⋮]=[rATrArATrB rBTrArBTrB]. \mathbf{r}^\mathrm{T}\mathbf{r} = \begin{bmatrix} \cdots & \mathbf{r}_A & \cdots\ \cdots & \mathbf{r}_B & \cdots \end{bmatrix} \begin{bmatrix} \vdots & \vdots\ \mathbf{r}_A & \mathbf{r}_B\ \vdots & \vdots \end{bmatrix} = \begin{bmatrix} \mathbf{r}_A^\mathrm{T}\mathbf{r}_A & \mathbf{r}_A^\mathrm{T}\mathbf{r}_B\ \mathbf{r}_B^\mathrm{T}\mathbf{r}_A & \mathbf{r}_B^\mathrm{T}\mathbf{r}_B \end{bmatrix}.

So if each vector of observations in your data matrix has mean 0, you can calculate the covariance matrix as:

1n−1rTr \frac{1}{n-1}\mathbf{r}^\mathrm{T}\mathbf{r}

Next Concept